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DETAILED ACTION 

Response to Amendment 

1 . Claims 1 -27 are pending. Claims 1 -5, 1 3-1 7 and 25-27 (and thereby all 
dependent claims) are amended. 

Response to Arguments 

2. Applicant's replacement Declaration, filed February 22, 2010, is accepted. 

3. Applicant's replacement Fig. 3, filed February 22, 2010, is accepted. 

4. Applicant's arguments filed February 22, 2010 with regard to the art rejections 
have been fully considered but they are not persuasive. 

5. Applicant's arguments with respect to claims 1-27 have been considered but are 
moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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7. Claims 1-5, 7-9, 11, 14-17, 19-21, 23, 26 and 27 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Apostolopoulos et al. (US 6404814 B1) in view of 
Jojic et al. (US 20040095374 A1 ). 

Regarding claims 1, 14, 26, and 27, Apostolopoulos et al. disclose a system and 
computer implemented method of changing the size of a presentation of an image data 
stream provided in an image data format (picture signals, col. 1, lines 5-20) created by 
an object based compression application (predictively coded, object based, col. 1, lines 
5-20; MPEG, DCT, col. 1 , lines 25-40) the method comprising the steps of: obtaining an 
image data stream coded in a format comprising at least one object layer (foreground 
and background object or sprite, col. 43, lines 10-15) and a background sprite layer 
including information about background elements of a scene displayed in a number of 
frames (background may be common to many pictures, col. 15, lines 30-40; background 
that does not change for a number of scenes, col. 43, lines 40-45), the image data 
stream having a first original field of view to be presented in (current picture, col. 24, 
lines 20-25), selecting at least parts of the image data stream outside the original field of 
view from the background sprite layer to obtain selected image data comprising values 
of pixel regions from an area larger than the original field of view (only one object is 
visible at a given pixel of the picture, col. 27, lines 5-10; pixels of the boundary tile are 
extrapolated pixels, col. 33, lines 15-25; scene that is larger than the scene described in 
any of the pictures in the group, panorama, col. 24, lines 5-25; camera pans, col. 43, 
lines 15-20), and changing the field of view by calculating an image to be displayed 
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conforming to a second field of view based on the obtained data and values, such that 
image data comprises pixel values substantially covering the second field of view (pixels 
of the boundary tile are extrapolated pixels, col. 33, lines 15-25; scene that is larger 
than the scene described in any of the pictures in the group, panorama, current picture 
to enable instance of object in later coded picture, col. 24, lines 5-25; camera pans, 
translation or perspective transform, col. 43, lines 15-25). 

Apostolopoulos et al. do not explicitly disclose these are layers. 

Jojic et al. teach a method of changing the size of a presentation of an image data 
stream provided in an image data format created by an object based compression 
application (video coding [0373]), the method comprising the steps of: obtaining an 
image data stream coded in a format comprising at least one object layer (layers, 
[0004]; objects such as car, truck, [0005]; sprite representing person, [0392]) and a 
background sprite layer including information about background elements of a scene 
displayed in a number of frames (portion of the background that is inferred from each 
frame, [0391]; reconstructed background provides a panoramic view of the scene in 
each reconstructed frame, [0392]), the image data stream having a first original field of 
view to be presented in (original frame, [0392]), selecting at least parts of the image 
data stream outside the original field of view from the background sprite layer to obtain 
selected image data comprising values of pixel regions from an area larger than the 
original field of view (obstructed pixels, [0016]; background pixels occluded, [0020]; 
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number of pixels preserved, background model to be "filled in", [0022]; occluded pixels, 
[0369]; background sprite is a panorama larger than the input images, [0393]), and 
changing the field of view by calculating an image to be displayed conforming to a 
second field of view based on the obtained data and values, such that image data 
comprises pixel values substantially covering the second field of view (number of pixels 
preserved, background model to be "filled in", [0022]; fill in background, [0391]; 
panorama built where original frames were not panoramic, [0392]; panorama larger, 
[0393]). 

Apostolopoulos et al. and Jojic et al. are in the same art of analysis of objects in video 
sequences (Apostolopoulos et al., col. 1, lines 5-30; Jojic et al., [0002]). The 
modification of Jojic et al. makes clear the actions described by Apostolopoulos et al. 
are taking place on objects and backgrounds in layers. It would have been obvious at 
the time of the invention to one of ordinary skill in the art to combine the inventions of 
Apostolopoulos et al. and Jojic et al., as this is a conventional scheme (Jojic, [0004]), 
and as Jojic et al. teach this allows for automatically and dynamically decomposing a 
video sequence into a number of layers, with each layer representing either an object or 
a background image, over each frame of the video sequence, capable of identifying 
sprites or objects of any geometry, including those with dynamic or changing 
geometries through a sequence of images without the need to use object specific 
models, reliably identifying sprites having unknown shapes and sizes which must be 
distinguished from the background, other sprites, sensor noise, lighting noise, and 
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significant amounts of deformation, and capable of processing large data sets in real 
time, or near-real time ([001 1]). 

Regarding claims 2 and 15, Apostolopoulos et al. and Jojic et al. disclose the invention 
of claim 1 and 14. Jojic et al. further teach the step of changing the field of view 
comprises combining objects of at least some of the layers of the decoded image data 
stream including said background sprite layer for providing an output data stream 
allowing presentation of image data (layers are combined using the sprite masks, layers 
are indexed, with a first or bottom layer representing the background layer, and each 
subsequent layer through the last layer representing layered sprites, [0023], [0092]; 
each layer is subject to a transformation, and then the layers are combined so that the 
frontal layers occlude the background layers [0219], [0233]; various combinations of the 
number of layers, [0374]). 

Regarding claim 3, Apostolopoulos et al. and Jojic et al. disclose the invention of claim 
1 . Apostolopoulos et al. and Jojic et al. further disclose displaying at least some of the 
image data in the stream on a display with the second field of view (Apostolopoulos et 
al., form the picture for display, col. 18, lines 20-30; Jojic et al., monitor or other display, 
[0074]). 
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Regarding claims 4 and 16, Apostolopoulos et al. and Jojic et al. disclose the invention 
of claims 1 and 14. Apostolopoulos et al. further disclose the coded stream is an MPEG- 
4 image data stream (col. 2, lines 5-15, lines 50-65; col. 12, lines 35-60). 

Regarding claims 5 and 17, Apostolopoulos et al. and Jojic et al. disclose the invention 
of claims 1 and 14. Apostolopoulos et al. and Jojic et al. further indicate processing the 
selected image data regarding mapping of less satisfactory positions of pixels in the 
second field of view (Apostolopoulos et al., changing one or both of the scale and 
orientation of one or more of the objects or otherwise mapping the objects onto the 
block-based picture, where objects overlap, mask and merge operations are performed, 
col. 27, lines 1-10, background object may move with a simple translation or perspective 
transform as the camera pans, col. 43, lines 15-25; Jojic et al., image is formed or 
"reconstructed" by composing a translated sprite appearance map with a background 
appearance map, and using a translated mask and its inverse, [0056], image sequence 
with motion blur artifacts, fill in the background sprite model, portion of the background 
inferred from each frame, reconstructed background provides panoramic view, [0390]- 
[0393]). 

Regarding claims 7 and 19, Apostolopoulos et al. and Jojic et al. disclose the invention 
of claims 5 and 17. Apostolopoulos et al. and Jojic et al. do not explicitly disclose the 
step of processing comprises cutting and pasting older picture material to later picture 
material if no or insufficient pixel regions outside the original field of view are at hand for 
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provision in the second field of view. However, as Apostolopoulos et al. indicate a 
previously-coded picture can be extrapolated to fill the bounding rectangle (col. 14, lines 
60-68; identifying an object not visible in the current picture but an instance is visible in 
a later coded picture (col. 23, lines 55-65), this indicates the possibility of copying 
content from a previous frame to a later frame. Further, Jojic et al. indicate from one bird 
model, an entire flock of birds in a composite bird can be created ([0370]) and teach 
copying snow ([0378]), and indicate a memory ([0066], [0075]), this indicates Jojic is 
teaching cutting and pasting older picture material to later picture material if no or 
insufficient pixel regions outside the original field of view are at hand for provision in the 
second field of view. 

Regarding claims 8 and 20, Apostolopoulos et al. and Jojic et al. disclose the invention 
of claims 5 and 1 7. Apostolopoulos et al. and Jojic et al. further disclose applying a 
geometrical image transformation for at least a region of the image outside of the 
original field of view where pixels are missing (Apostolopoulos et al., perspective 
transform as the camera pans, col. 43, lines 15-25; Jojic et al., sprite mask and the 
sprite appearance map are transformed before being composed with the background, 
[0021], [0022], given transforms of scale, rotation, and x-y translations for a particular 
sprite, [0364]) 

Regarding claims 9 and 21, Apostolopoulos et al. and Jojic et al. disclose the invention 
of claims 5 and 17. Apostolopoulos et al. further disclose filling missing pixels using 



Application/Control Number: 10/597,563 Page 9 

Art Unit: 2624 

extrapolation (the amplitude of the object inside the boundary tile may be first 
extrapolated to fill the tile, col. 14, lines 15-25). 

Regarding claims 11 and 23, Apostolopoulos et al. and Jojic et al. disclose the method 
and device of claims 5 and 17. Apostolopoulos et al. and Jojic et al. further indicate the 
step of processing comprises shifting at least a region of the pixels of one layer in 
relation to the pixels of at least one other layer in order to allow the objects of said one 
layer to be adjusted in relation to objects of said other layer (Apostolopoulos et al., shift 
tile to construct current picture, col. 22, lines 35-45; Jojic et al., occluded objects 
modeled using layers of sprites, translated sprite maps with a background appearance 
map, [0260]-[0266]). 

8. Claims 6, 13, 18 and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Apostolopoulos et al. (US 6404814 B1) in view of Jojic et al. (US 
20040095374 A1) as applied to claim 2, 5, 14 and 17 above, further in view of Volk (US 
5673401). 

Regarding claims 6 and 18, Apostolopoulos et al. and Jojic et al. disclose the invention 
of claims 5 and 17. Apostolopoulos et al. and Jojic et al. do not explicitly disclose the 
step of processing comprises any of the steps of stretching the image in one direction, 
stretching the image in one direction with uneven zoom factor, stretching the image in 
two directions or providing black bars at the sides of the image. 
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Volk indicates the step of processing comprises any of the steps of stretching the image 
in one direction, stretching the image in one direction with uneven zoom factor, 
stretching the image in two directions or providing black bars at the sides of the image 
(Volk, device independent bitmap can be stretched, col. 33, lines 45-55). 

Apostolopoulos et al. and Jojic et al. and Volk are in the same art of analysis of objects 
in video sequences (Apostolopoulos et al., col. 1 , lines 5-30; Jojic et al., [0002]; Volk, 
col. 9, lines 45-55, col. 10, lines 30-50). The combination of Volk would allow objects to 
be stretched. It would have been obvious at the time of the invention to one of ordinary 
skill in the art to combine the stretching of Volk with the invention of Apostolopoulos et 
al. and Jojic et al., because this is one of a limited number of operations that can be 
performed on an image, Volk indicates the invention provides for a customizable sprite- 
based graphical user interface (col. 6, lines 65-68), and would have been obvious to try. 

Regarding claims 13 and 25, Apostolopoulos et al. and Jojic et al.disclose the method 
and device according to claims 2 and 14. Apostolopoulos et al. and Jojic et al. do not 
explicitly disclose the values of pixel regions outside the first field of view are provided in 
at least one different output data stream than the stream including the combined 
objects. 
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As Volk discloses an interactive network in which the system is used for delivering 
information and receiving instructions via a "two-way" distribution network (col. 11, lines 
10-15), this is good evidence that a different data stream is used for the different fields 
of view. 

9. Claims 10 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Apostolopoulos et al. (US 6404814 B1) in view of Jojic et al. (US 20040095374 A1) 
as applied to claims 8 and 20 above, further in view of Kaye et al. (US 20050104878 
A1). 

Regarding claims 10 and 22, Apostolopoulos et al. and Jojic et al. disclose the method 
and device according to claims 8 and 20. Apostolopoulos et al. and Jojic et al. do not 
explicitly disclose the geometrical image transformation comprises copying border 
pixels for filling missing pixels. 

Kaye et al. indicate a missing gap of pixels can be filled by repeating pixels from the 
edge of the background object (abstract, [0004], [0035]). 

Apostolopoulos et al. and Jojic et al. and Kaye are in the same art of sequences of 
images (Apostolopoulos et al., col. 1, lines 5-30; Jojic et al., [0002]; Kaye, abstract). The 
combination of Kaye would allow border pixels to be replicated to fill in missing pixels, 
rather than just relying on pixels within the object boundary as taught by Apostolopoulos 
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et al. It would have been obvious at the time of the invention to one of ordinary skill in 
the art to use the method of filling in the region as taught by Kaye et al. with the 
invention of Apostolopoulos et al. and Jojic et al., as this is a technique conventionally 
used to fill in unknown regions, and would have been one of a limited number of ways to 
fill in the information. 

10. Claims 12 and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Apostolopoulos et al. (US 6404814 B1) in view of Jojic et al. (US 20040095374 A1) 
as applied to claims 1 and 14 above, further in view of Alattar et al. (US 20030185417 
A1) further in view of Garrido et al. (US 20040022318 A1). 

Regarding claims 12 and 24, Apostolopoulos et al. and Jojic et al. disclose the method 
and device according to claims 1 and 14. Apostolopoulos et al. and Jojic et al. do not 
explicitly disclose an aspect ratio change from 4:3 to 16:9. 

Alattar et al. teach in many video applications, frames of video may be re-sized 
horizontally and vertically (e.g., by aspect ratio changes) ([0027]). 

Apostolopoulos et al. and Jojic et al. and Alattar et al. are in the same art of video 
(Apostolopoulos et al., col. 1 , lines 5-30; Jojic et al., [0002]; Alattar, abstract). The 
modification of Alattar makes explicit changes in aspect ratio. It would have been 
obvious at the time of the invention to one of ordinary skill in the art to combine the art 
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of Alattar et al. with that of Apostolopoulos et al. and Jojic et al., as this is one of a 
limited number of ways to alter an image frame, Alattar et al. teach this is done in many 
video applications ([0207]), would increase the flexibility of viewing options, and would 
have been obvious to try. 

Apostolopoulos et al. and Jojic et al. and Alattar et al. do not explicitly disclose an 
aspect ratio change from 4:3 to 16:9. 

Garrido et al. teach professional video engineering organizations collaborated on the 
creation of the signal representation standard on a 4:3 aspect ratio monitor, where 
applications such as DVD later diluted the same pixel grid to cover a one third, wider 
screen area, thus the horizontal density on 16:9 anamorphic DVD titles is one third less 
than standard 4:3 "pan & scan" titles ([0047]). 

Apostolopoulos et al. and Jojic et al. and Alattar et al. and Garrido et al. are in the same 
art of video (Apostolopoulos et al., col. 1, lines 5-30; Jojic et al., [0002]; Alattar, abstract; 
Garrido et al., abstract). The modification of Garrido et al. inserts specific values for the 
aspect ratio change. It would have been obvious to one having ordinary skill in the art at 
the time the invention was made to operate at these values, as Garrido et al. indicate 
these were taken into consideration by professional organizations ([0047]), indicating 
these are conventional values, and since it has been held that discovering an optimum 
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value of a result effective variable involves only routine skill in the art. In re Boesch, 617 
F.2d 272, 205 USPQ 215 (CCPA 1980). 

Conclusion 

1 1 . THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to MICHELLE ENTEZARI whose telephone number is 
(571)270-5084. The examiner can normally be reached on M-Th, 7:30am-5pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on (571)272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Michelle Entezari/ 
Examiner, Art Unit 2624 
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